Sensitive Error Correcting Output Codes

We present a reduction from cost sensitive classi cation to binary classi cation based on (a modi cation of) error correcting output codes. The reduction satis es the property that regret for binary classi cation implies l2-regret of at most 2 for cost-estimation. This has several implications: 1) Any regret-minimizing online algorithm for 0/1 loss is (via the reduction) a regret-minimizing online cost sensitive algorithm. In particular, this means that online learning can be made to work for arbitrary (i.e. totally unstructured) loss functions. 2) The output of the reduction can be thresholded so regret for binary classi cation implies at most 4 √ Z regret for cost sensitive classi caiton where Z is the expected sum of costs. 3) Using the canonical embedding of multiclass classifcation into cost sensitive classi cation, this reduction shows that binary regret implies l2-regret of at most 4 in the estimation of class probabilities. For a hard prediction, this implies at most 4 √ multiclass regret.